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Abstract 

We study the large deviations performance, i.e., the exponential decay rate of the error probability, of 
distributed detection algorithms over random networks. At each time step k each sensor: 1) averages its 
decision variable with the neighbors decision variables; and 2) accounts on-the-fly for its new observation. 
We show that distributed detection exhibits a "phase change" behavior. When the rate of network 
information flow (the speed of averaging) is above a threshold, then distributed detection is asymptotically 
equivalent to the optimal centralized detection, i.e., the exponential decay rate of the error probability 
for distributed detection equals the Chernoff information. When the rate of information flow is below a 
threshold, distributed detection achieves only a fraction of the Chernoff information rate; we quantify this 
achievable rate as a function of the network rate of information flow. Simulation examples demonstrate 
our theoretical findings on the behavior of distributed detection over random networks. 

Keywords: Chernoff information, distributed detection, random network, running consensus, informa- 
tion flow, large deviations. 
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I. Introduction 

Existing literature on distributed detection can be broadly divided into three different classes. The first 
studies parallel (fusion) architectures, where all sensors transmit their measurements, or local likelihood 
ratios, or local decisions, to a fusion node; the fusion node subsequently makes the final decision (see, 
e.g., [1], [2], [3], [4].) The second considers consensus-based detection, where no fusion node is required, 
and sensors communicate with single-hop neighbors only over a generic network (see, e.g., [5], [6]). 
Consensus-based detection operates in two phases. First, in the sensing phase, each sensor collects 
sufficient observations over a period of time. In the second, communication phase, sensors subsequently 
run the consensus algorithm to fuse their local log likelihood ratios. More recently, a third class of 
distributed detection has been proposed (see [7], [8], [9],) where, as with consensus-based detection, 
sensors communicate over a generic network, and no fusion node is required. Differently than consensus- 
based detection, sensing and communication phases occur in the same time step. Namely, at each time 
step k, each sensor: 1) exchanges its current decision variable with single-hop neighbors; and 2) processes 
its new observation, gathered at time step k. 

In this paper, we focus on the third class of distributed detection, and we provide fundamental analysis 
of the large deviation performance, i.e., of the exponential decay rate of the error probability (as k — > oo,) 
when the underlying communication network is randomly varying. Namely, we show that distributed 
detection over random networks is asymptotically equivalent to the optimal centralized detection, if the 
rate of information flow (i.e., the speed of averaging) across the random network is large enough. That 
is, if the rate of information flow is above a threshold, then the exponential rate of decay of the error 
probability of distributed detection equals the Chernoff information-the best possible rate of the optimal 
centralized detector. When the random network has slower information flow (asymptotic optimality cannot 
be achieved,) we find what fraction of the best possible rate of decay of the error probability distributed 
detection can achieve; hence, we quantify the tradeoff between the network connectivity and achievable 
detection performance. 

Specifically, we consider the problem where sensors cooperate over a network and sense the environ- 
ment to decide between two hypothesis. The network is random, varying over time k (see, e.g., [10]); in 
alternative, the network uses a random communication protocol, like gossip (see, e.g., [11]). The network 
connectivity is described by {W(k)}'^L 1 , the sequence of identically distributed (i.i.d.) consensus weight 
matrices. The sensors' observations are Gaussian, correlated in space, and uncorrelated in time. At each 
time k, each sensor: 1) communicates with its single-hop neighbors to compute the weighted average of 
its own and the neighbors' decision variables; and 2) accounts for its new observation acquired at time 
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k. The network's rate of information flow (i.e., the speed of averaging,) is then measured by | log r | , 
where r is the second largest eigenvalue of the expected value of W(k) 2 . We then show that distributed 
detection exhibits a "phase change" behavior. If | log r| exceeds a threshold, then distributed detection 
is asymptotically optimal. If | log r| is below the threshold, then distributed detection achieves only a 
fraction of the best possible rate; and we evaluate the achievable rate as a function of r. Finally, we 
demonstrate by simulation examples our theoretical findings on the behavior of distributed detection over 
random networks. 

Several recent references [8], [9], [7] consider different variants of distributed detection of the third 
class. We consider in this paper the running consensus, the variant in [7]. 

In the context of estimation, distributed iterative schemes have also been proposed. References [12], 
[13] propose diffusion type LMS and RLS algorithms for distributed estimation; references [14], [15] 
also propose algorithms for distributed estimation, based on the alternating direction method of multipli- 
ers. Finally, reference [16] proposes stochastic- approximation type algorithm for distributed estimation, 
allowing for randomly varying networks and generic (with finite second moment) observation noise. With 
respect to the network topology and the observation noise, we also allow for random networks, but we 
assume Gaussian, spatially correlated observation noise. 

We comment on the differences between this work and reference [7], which also studies asymptotic 
performance of distributed detection via running consensus, with i.i.d. matrices W(k). Reference [7] 
studies a problem different than ours, in which the means of the sensors' observations under the two 
hypothesis become closer and closer; consequently, there is an asymptotic, non zero, probability of miss, 
and asymptotic, non zero, probability of false alarm. Within this framework, the running consensus 
achieves the efficacy [17] of the optimal centralized detector, under a mild assumption on the underlying 
network being connected on average. In contrast, we assume that the means of the distributions do not 
approach each other as k grows, but stay fixed with k. The Bayes error probability exponentially decays 
to zero, and we examine its rate of decay. We show that, in order to achieve the optimal decay rate of 
the Bayes error probability, the running consensus needs an assumption stronger than connectedness on 
average, namely, the averaging speed needs to be sufficiently large (as measured by | logr|.) 

In recent work [18], we considered running consensus detection when the underlying network is 
deterministically time varying; we showed that asymptotic optimality holds if the graph that collects 
the union of links that are online at least once over a finite time window is connected. In contrast, we 
consider here the case when the underlying network or the communication protocol are random, and we 
establish a sufficient condition for optimality in terms of averaging speed (measured by | logr|.) 
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Paper organization. The next paragraph defines notation that we use throughout the paper. Section II 
reviews standard asymptotic results in hypothesis testing, in particular, the Chernoff lemma. Section III 
explains the sensor observations model that we assume and studies the optimal centralized detection, as 
if there was a fusion node to process all sensors' observations. Section IV presents the running consensus 
distributed detection algorithm. Section V studies the asymptotic performance of distributed detection on a 
simple, yet illustrative, example of random matrices W(k). Section VI studies asymptotic performance of 
distributed detection in the general case. Section VII demonstrates by simulation examples our theoretical 
findings. Finally, section VIII concludes the paper. 

Notation. We denote by: Aij or [A]^ (as appropriate) the (i,j)-th entry of a matrix A; ai or [a]i the z-th 
entry of a vector a; /, 1, and e«, respectively, the identity matrix, the column vector with unit entries, 
and the i-th column of /, J the N x N matrix J := (l/iV)ll T ; || • ||/ the vector (respectively, matrix) 
/-norm of its vector (respectively, matrix) argument, || • || = || • H2 the Euclidean (respectively, spectral) 
norm of its vector (respectively, matrix) argument, || • \\p the Frobenius norm of a matrix; Aj(-) the i-th 
largest eigenvalue, Diag (a) the diagonal matrix with the diagonal equal to the vector a; E [•] and P (•) the 
expected value and probability, respectively; X4 the indicator function of the event A; finally, Q(-) the 
Q-function, i.e., the function that calculates the right tail probability of the standard normal distribution; 



This section reviews standard asymptotic results in hypothesis testing, in particular, the Chernoff 
lemma, [19]; it also introduces certain inequalities for the Q(-) function that we use throughout. We 
first formally define the binary hypothesis testing problem and the log-likelihood ratio (LLR) test. 
Binary hypothesis testing problem: Log-likelihood ratio test. Consider the sequence of independent 
identically distributed (i.i.d.) <i-dimensional random vectors (observations) y(k), k = 1,2,..., and the 
binary hypothesis testing problem of deciding whether the probability measure generating y(k) is vq 
(under hypothesis Ho) or v\ (under Hi). Assume that v\ and vq are mutually absolutely continuous, 
distinguishable measures. Based on the observations y(l), y(k), formally, a decision test T is a 
sequence of maps : M. kd — > {0,1}, k = 1,2,..., with the interpretation that T^{y{l), y{k)) = I 
means that Hi is decided, / = 0, 1. Specifically, consider the log-likelihood ratio (LLR) test to decide 
between Hq and Hi, where the is given as follows: 




II. Preliminaries 




k 



(1) 



3=1 

Tk = l{v(k)>y k } 



(2) 
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where L(k) := log ^ (y(k)) is the LLR (given by the Radon-Nikodym derivative of v\ with respect to 
uq evaluated at y(k),) and 7^ is a chosen threshold. 

Asymptotic Bayes detection performance: Chernoff lemma. Given a test T, we are interested in 
quantifying the detection performance, namely, in determining the Bayes error probability after k data 
(observation) samples are processed: 

P e (k) = P(H )a(k)+P(H 1 )(3(k), (3) 

where P (Hi) are the prior probabilities, a(k) = P(T k = 1\H ) and /3(k) = P(T k = 0\Hi) are, respec- 
tively, the probability of false alarm and the probability of a miss. Generally, exact evaluation of a(k) 
and f3(k) (and hence, P e (k)) is very hard (as in the case of distributed detection over random networks 
that we study; see also [20] for distributed detection on a parallel architecture.) We seek computationally 
tractable estimates of P e (k), when k grows large. Typically, for large k, P e (k) is a small number (i.e., 
the detection error occurs rarely,) and, in many models, it exponentially decays to zero as k — > +00. 
Thus, it is of interest to determine the (large deviations) rate of exponential decay of P e (k), given by: 

lim -- log P e {k). (4) 

fe^oo k 

Lemma 1 ([19], [21]) states that, among all possible decision tests, the LLR test with zero threshold 
maximizes (4) (i.e., has the fastest decay rate of P e (k).) The corresponding decay rate equals the Chernoff 
information C, i.e., the Chernoff distance between the distributions of y(k) under Hq and Hi, where C 
is given by, [19]: 



Lemma 1 (Chernoff lemma) If P(Hq) G (0,1), then: 

sup { limsup — - logP e (k) 1 = C, (6) 
where the supremum over all possible tests T is attained for the LLR test with 7^. = 0, VTc. 
Asymptotically optimal test. We introduce the following definition of the asymptotically optimal test. 

Definition 2 The decision test T is asymptotically optimal if it attains the supremum in eqn. (6). 

We will find a necessary condition and a sufficient condition for asymptotic optimality (in the sense of 
Definition 2) of the running consensus distributed detection. 
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Inequalities for the standard normal distribution. We will use the following property of the Q(-) 
function, namely, that for any t > (e.g., [22]): 



We proceed with the Gaussian model for which we find (in section V) conditions for asymptotic 
optimality of the running consensus distributed detection. Subsection III-A describes the model of the 
sensor observations that we assume. Subsection III-B describes the (asymptotically) optimal centralized 
detection, as if there was a fusion node that collects and processes the observations from all sensors. 

A. Sensor observations model 

We assume that iV sensors are deployed to sense the environment and to decide between the two 
possible hypothesis, H\ and Hq. Each sensor i measures a scalar quantity yi(k) at each time step k; 
all sensors measure at time steps k = 1,2,... Collect yi(k) , s, i = l,...,N, into N x 1 vector y(k) = 
(yi(k), j/jv(&)) • We assume that {y(k)} has the following distribution: 



The quantity mi is the constant signal; the quantity £(fc) is zero-mean, Gaussian, spatially correlated 
noise, i.i.d. across time, with distribution ((k) ~ Af(0,S), where S is a positive definite covariance 
matrix. Spatial correlation of the measurements (i.e., non-diagonal covariance matrix S) accounts for, 
e.g., dense deployment in sensor networks. 

B. (Asymptotically) optimal centralized detection 

This subsection studies optimal centralized detection under the Gaussian assumptions in III-A, as if 
there was a fusion node that collects and processes all sensors' observations. The LLR decision test is 
given by eqns. (1) and (2), where it is straightforward to show that now the LLR takes the following 
form: 




(7) 



III. Centralized detection 



Under H t : y(k) = mi + ((k), 1 = 0,1. 



(8) 




(9) 




(10) 



(11) 
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Define the vector v G R as 

v := 5 _1 (mi -m ). (12) 
Then, the LLR L(/c) can be written as follows: 

L(k) = f> L{k) - i^M) = f>(*) (13) 

i=l v y i=l 

Thus, the LLR at time k is separable, i.e., the LLR is the sum of the terms r)i(k) that depend affmely on 
the individual observations yi(k). We will exploit this fact in section IV to derive the distributed, running 
consensus, detection algorithm. 

Bayes probability of error: finite number of observations. The minimal Bayes error probability, 
P c e en (/c), when k samples are processed, and P(Hq) = P(H\) = \ (equal prior probabilities), is attained 
for the (centralized) LLR test with zero threshold; P c e en (fc) equals: 

The quantity -P c e en (fc) will be of interest when we compare (by simulation, in Section VII) the running 
consensus detection with the optimal centralized detection, in the regime of finite k. 

Bayes probability of error: time asymptotic results. The Chernoff lemma (Lemma 1) applies also to 
the (centralized) detection problem as defined in subsection III-A. It can be shown that the Chernoff 
information, in this case, equals: 

C = C to t = \{mi - m ) T S~ 1 (m 1 - m Q ). (15) 
o 

In eqn. (15), the subscript tot designates the total Chernoff information of the network, i.e., the Chernoff 
information of the observations collected from all sensors. Specifically, if the sensor observations are 
uncorrelated (the noise covariance matrix S = Diag(af, ...,cr^),) then: 

Ctot = £K_^ = £ Ci _ (]6) 

i=l 1 i=l 

where C, is the Chernoff information of the individual sensor i. That is, Cj equals the best achievable 
rate of the Bayes error probability, if the sensor i worked as an individual (it did not cooperate with the 
other sensors.) 

Lemma 3 (Chernoff lemma for asymptotically optimal centralized detector) Consider the observation 
model defined in subsection III-A, and let P(Hq) G (0, 1). The LLR test with 7^ = 0, \/k, is asymptoti- 
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cally optimal in the sense of Definition 2. Moreover, for the LLR test with 7^ = 0, Mk, we have: 



lim ilogP c e cn (A:) = -C tot , (17) 



where C to t is given by eqn. (15). 



IV. Distributed detection 



We now consider distributed detection, under the same assumptions on the sensor observations as 
in III-A; but the fusion node is no longer available, and the sensors cooperate through a randomly 
varying network. Specifically, we consider the running consensus distributed detection, proposed in [7], 
and we extend it to spatially correlated observations. At each time k, each sensor i improves its decision 
variable, call it Xi(k), two-fold: 1) by exchanging the decision variable locally with its neighbors and 
computing the weighted average of its own and the neighbors' variables; and 2) by incorporating its new 
observation at time k. 

Recall the definition of the vector v in eqn. (12) and the scalar i]i(k) in eqn. (13). The update of Xi(k) 
is then as follows: 



k 

Xiik + l) = 



^— W u {k) Xi {k) + £ Wij(k)xj(k) + 1 ~^Vi(k + 1), k = 1, ... (18) 
= »7i(l)- 

Here Oj(fc) is the (random) neighborhood of sensor i at time k, and Wij(k) are the (random) averaging 
weights. 1 The local sensor z's decision test at time k, T^j, is given by: 

Tk,i ■■= X{ :E .( fc ) > o}, (19) 

i.e., Hi (resp. Hq) is decided when Xi(k) > (resp. Xi(k) < 0.) Let x(k) = (xi(k), X2(k), xat(/c)) t 
and 7]{k) = (r/i(/c), r/jv(/c)) T . Also, collect the averaging weights Wij(k) in N x N matrix W(k), 
where, clearly, Wij(k) = if the sensors i and j do not communicate at time step k. The algorithm in 
matrix form becomes: 

x(k + l) = -^-W(k)x(k) + -^— V (k + l),k = l,... (20) 
k + 1 k + 1 

x{\) = V (l). 

'We remark that, to implement the algorithm, sensor i has to know the quantities «; := [S _1 (mi — mo)] ., [m\]i and [mo]^; 
this knowledge can be acquired in the training period of the sensor network. 
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We remark that the algorithm in (20) extends the running consensus algorithm in [23] for spatially 
correlated sensor observations (non-diagonal covariance matrix S.) When S is diagonal, the algorithm 
in (20) reduces to the algorithm in [23]. 2 

We allow the averaging matrices W(k) to be random. Formally, let (J), P) be a probability space 
(where SI is a sample space, J 7 is a cr-algebra, and P : T — > [0, 1] is a probability measure.) For 
any k, W(k) is a random variable, i.e., an ^-measurable function W(k) = W(co; k), u G Q, W(k) : 
f2 — > ~R NxN . We now summarize the assumptions on W(k). Recall that J := -^(11 T ) and denote by 
W(k) := W(k) — J. From now on, we will drop the index k from W(k) and W(k) when we refer to 
the distribution of W{k) and W(k). 

Assumption 4 For the sequence of matrices {W(k)}^ =l , we assume the following: 

1) The sequence {W(k)} r ^ =1 is i.i.d. 

2) W is symmetric and stochastic (row-sums are equal to 1 and the entries are nonnegative,) with 
probability one. 

3) The random matrix W(l) and the random vector y(s) are independent, V/, Vs. 

In sections V and VI, we examine what (additional) conditions the matrices W(k) have to satisfy, to 
achieve asymptotic optimality of the distributed detection algorithm. 

Network supergraph. Define also the network supergraph as a pair G := (V, E), where V is the set 
of nodes with cardinality |V| = N, and E is the set of edges with cardinality \E\ = M, defined by: 
E = {{i,j} : P (Wij / 0) > 0, i < j}. Clearly, when, for some {i, j}, P (Wij / 0) = 0, then the link 
{i, j} E and nodes % and j never communicate. 
For subsequent analysis, it will be useful to define the matrices &(k,j), for k > j > 1, as follows: 

$(k,j) := W{k - l)W(k - 2)...W{j). (21) 

Then, the algorithm in eqn. (20) can be written as: 

1 k ~ l 1 
x (k) = -Y J ^,j)v(j) + ^(k), fc = 2,3,... (22) 

Also, introduce: 

${k, j) := W(k - l)W(k - 2)...W(j), k > j > 1, (23) 

2 Another minor difference between [23] and eqn. (20) is that [23] multiplies the log-likelihood ratio term (the term analogous 
to -j^rjri(k + 1)) by iV; this multiplication does not affect detection performance. 
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and remark that 

*(k,j) = $(k,j)-J. 

Recall the definition of the N x 1 vector v in (12). The sequence of N x 1 random vectors {rj(k)}, 
conditioned on Hi, is i.i.d. The vector rj(k) (under hypothesis Hi, I = 0,1) is Gaussian with mean rr$ 
and covariance S v : 

m (0 = (-l)(' +1 )Diag(v) -(mi -m ) (24) 
S r > = Diag (v) SDiag (v) . (25) 

Here Diag(v) is a diagonal matrix with the diagonal entries equal to the entries of v. 

V. Asymptotic performance of distributed detection: Switching fusion example 

In this section, we examine asymptotic performance of distributed detection algorithm on a simple and 
impractical, yet illustrative example; we tackle the generic case in Section VI. The network at a time 
step k can either be fully connected, with probability p, or completely disconnected (without edges,) with 
probability 1 — p. Specifically, the distribution of the random averaging matrix W(k) is given by: 



W(k) 



J with prob. p 

(26) 

/ with prob. 1 — p. 



With model (26), at each time step k, each sensor behaves as a fusion node, with probability p, and as an 
individual detector, with probability 1 — p. We call this communication model the switching fusion. We 
show that, to achieve asymptotic optimality of distributed detection, the fusion step (W(k) = J) should 
occur sufficiently often, i.e., p should exceed a threshold. Namely, we find necessary and sufficient 
condition for the asymptotic optimality in terms of p. When distributed detection is not optimal (p is 
below the threshold,) we find the achievable rate of decay of the error probability, as a function of p. 
The goal of the switching fusion example is two-fold. First, it provides insight on how the amount of 
communication (measured by p) affects detection performance. Second, it explains in a clear and natural 
way our methodology for quantifying detection performance on generic networks (in Section VI.) Namely, 
Section VI mimics and extends the analysis from Section V to derive distributed detection performance 
on generic networks. We next detail the sensor observations model. 

We assume that the observations yi(k) of N different sensors are uncorrected, and that the individual 
Chernoff information, given by eqn. (16), is the same at each sensor i. Hence, we have C to t = N Ci. We 
assume that, at time instant k, the network can either be fully connected, with probability p, or without 
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edges, with probability I — p. 

Denote by Pf&JJk) the Bayes error probability at sensor i, after k samples are processed. We have 
the following Theorem on the asymptotic performance of the distributed detection algorithm. 

Theorem 5 Consider the distributed detection algorithm given by eqns. (18) and (19). Assume that the 
sensor observations are spatially uncorrected and that the Chernoff information Q is equal at each sensor 
i. Let W(k) be i.i.d. matrices with the distribution given by eqn. (26). Then, the exponential decay rate 
of the error probability is given by: 



Qot if |log(l-p)| > CtotC/V-1 

1 

> ^sMC^ _M=p)[ otherwise. 



1 



Moreover, a necessary and sufficient condition for asymptotic optimality, in the sense of Definition 2, 
is given by: 

|l0g(1 -^>C tot = iVC, (28) 



N - : 

Condition (28) says that the network connectivity should be good enough (i.e., p should be large 
enough,) in order to achieve the asymptotic optimality of distributed detection. Also, there is a "phase 
change" behavior, in a sense that distributed detection is asymptotically optimal above a threshold on p, 
and it is not optimal below that threshold. Further, we can see that, as p decreases, distributed detection 
performance becomes worse and worse, and it approaches the performance of an individual sensor- 
detector. (See Figure 1, and eqn. (27).) 

We proceed with proving Theorem 5. In Section VI, we will follow a reasoning similar to the proof 
of Theorem 5 to provide a sufficient condition for asymptotic optimality on generic networks. 

Proof of Theorem 5: First, remark that x(k), conditioned on Ho, is equal in distribution to —x(k), 
conditioned on H\. This is true because rj(k), conditioned on Hq, is equal in distribution to —r](k), 
conditioned on Hi, for all k; and the distribution of W(k) does not depend on the active hypothesis, Hq 
or Hi. Denote by P; (•) = F(-\Hi), I = 0, 1, and consider the probability of false alarm, the probability 
of miss, and the Bayes error probability at sensor i (with the running consensus detector,) respectively, 
given by: 

CMis(fc) = n (xi(k) > 0) , ft.disCfc) = Pi (xi(k) < 0) (29) 
^ dis (fc) = P(H )F (x i (k)>0) + P(H 1 )Fi(x i (k)<0). (30) 
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0.12 




Fig. 1. Exponential decay rate of error probability (f>* (given by eqn. (26)) for the model considered in Section V. The network 
has N = 20 sensors and Ctot = 0.1. The optimal rate (equal to C to t) is achieved for p > 0.83. 



Remark that P z (xi(k) = 0) = 0, / = 0, 1. Thus, we have that 

PiM k ) = p i (zi(*0 < 0) = Po (si(fc) > 0) = a iidi8 (fc), Vfc, Vi. (31) 
From eqns. (30) and (31), it can be shown that: 

limsup * logi^ dis (/c) = limsupiloga iidis (£;) (32) 
liminf - log P[ dis (k) = liminf y \oga idis (k). (33) 

We further assume that Hq is true, and we restrict our attention to ai^is(k), but the same conclusions 
(from (31)) will be valid for (3i :< a s (k) also. We now make the key step in proving Theorem 5, by defining 
a partition of the probability space Q. Fix the time step k and denote by Ai, I = 0, k — 1, the event 



A, 



{max {s G {1, — 1}: W(s) = J} = 1} for I = 1, k - 1 
{W(a) = I, s = 1, 1} foe 1 = 0. 

That is, ^4/ is the event that the largest time step s < k — 1, for which W(s) = J, is equal to s = I. 
(The event Ai includes the scenarios of arbitrary realizations of iy(s)-either J or /-for s < I; but it 
requires W(s) = / for all / < s < k — 1.) Remark that A\ is a function of k, but the dependence 
on k is dropped for notation simplicity. We have that F (Ai) = p(l —p)^ 1 ^ 1 , for / = 1, — 1, 
and IP (A)) = (1 — Also, each two events, Ai and Aj, j / /, are disjoint, and U^ 1 A\ = 17, 

i.e., the events A\, I = 0, ...,k — 1, constitute a finite partition of the probability space U. (Note that 
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Ef= lp (A) = 1.) Recall the definition of &(k,j) in eqn. (21) and note that, if A\ occurred, we have: 



I if k - 1 > s > I 
J if s < I. 



<£(£;, s) = < 

Further, conditioned on A[, we have that (when I = 0, first sum in eqn. (35) does not exist) 



\j=i 3=1+1 J 



K 3=l 7 3=1+1 

Hence, conditioned on A\, Xi(k) is a Gaussian random variable, 

Xi(k)\Ai ~ N (9(1; k), ( 2 (l;k)) , 

where 



Define 



and remark that 



where P (-) := P(-\H ). 

Using the total probability, we can write ai^is(k) = Po (x%(k) > 0) as: 

fe-i 



(34) 



(35) 



41 

9(l;k) = --Ctot-4(fc-0Ci = -4fcCi (36) 
8/ 

C 2 (/;fc) = — Ctot + 8(fc-Z)Ci. (37) 



X(i; fe : = -77TTT = / ( 38 ) 

-tjr + (k-i) 



P (xi(fc)>0|A,) = Q( x (Z;fc)), 



= ^P (^(fc)>0|^)P(A) (39) 

1=0 

fe-i 

= E 2(x(i; *0) P(i - p)^ 1 + G(x(0; fc)) (i - P)"" 1 . 

We now proceed with calculating the exponential rate of decay of ^is(k) (and hence, -Pf dis (fc)) as 
k — > oo. The key ingredient to do that is the representation of a^disC^) in eqn. (39), and the inequalities 
for the Q-function in eqn. (7). Namely, it can be shown that, when k grows large, ai^i s {k), and hence, 
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P i e dis (/c), behaves as (here we present the main idea but precise statements are in the Apendix): 

k— 1 

^di S (*o~E e-W) ' (4°) 

3=0 

where 

(f>(j;k):= — — + j|log(l-p)|. (41) 

Hence, the quantities <j)(j; k), for different j's, represent different "modes" of decay; the decay of Pf dis {k) 
is then determined by the slowest mode <f>(k), defined by: 

<j>(k)= min 4>(j;k). (42) 

J=0,...,fe-1 

More precisely, using eqns. (32), (33), the expression for ai^i s {k) in eqn. (39), and the inequalities (7), 
it can be shown that: 

liminf -I log Pf dis (k) > liminf£(fc) (43) 
limsup — - \ogP? dis (k) < limsup0(A;). 

fc^oo ™ fc-^oo 

The detailed proof of inequalities (43) is in the Appendix. 

We proceed by noting that the minimum of <f>(j; k) over the discrete set j G {0, 1, k — 1} does not 
differ much from the minimum of 4>(j;k) over the interval [0, k — 1]. Denote by 4>*(k) the minimum of 
4>(j; k) over [0, k — 1]: 

0*(fc) = min 00'; (44) 
je[o,fc-i] 

Then, it is easy to verify that: 

4>*(k) < 4>(k) < 4>*(k) (l + iV ^ 1 ) + |log(1 fc ~ p)l . (45) 

The function (f)(j, k) is convex in its first argument on j G [0, k — 1]; it is straightforward to calculate 
4>*(k), which can be shown to be equal to: 



cf>*(k) = { 



if |log(l-p)|>fe!i£ 



(i+^) 2 

— + *=* | log(l - p)| if I log(l - p)| < ^-'ff- 1 * (46) 



^ 2 / lioga-rtic^ _ IJ^l _ IJ^pA otherwise . 



December 22, 2010 



DRAFT 



The limit lim^oo (f>*(k) =: 0* exists, and is equal to: 



Ctot 

%t + |log(l-p)| 



N 

2 / |io g (i-p)|c tot 



I iog(i-p)| 

N-l 



if |log(l-p)| > C tot (iV -1) 
if |log(l-p)|<^-i) 

otherwise. 



From eqns. (45) and (47), we have: 



lim <f>(k) = lim <fi*(k) 

fe^oo k— >oo 
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(47) 



(48) 



In view of eqns. (48) and (43), it follows that the rate of decay of the error probability at sensor i is: 



lim --logP t e dis {k) = lim (j)(k) = 4>*. 

k^>oo K k— >oo 

The necessary and sufficient condition for asymptotic optimality then follows from eqn. (47). 



(49) 



VI. Asymptotic performance of distributed detection: General case 

This section provides a necessary condition, and a sufficient condition for asymptotic optimality of 
distributed detection on generic networks and for generic, spatially correlated, Gaussian observations. 
When distributed detection is not guaranteed to be optimal, this section finds a lower bound on the 
exponential decay rate of error probability, in terms of the system parameters. We start by pursuing 
sufficient conditions for optimality and evaluating the lower bound on the decay rate of the error 
probability. 



A. Sufficient condition for asymptotic optimality 

Recall that r := A 2 (E [W(/c) 2 ]) = ||E [W(/e) 2 ] - J||. It is well known that the quantity r measures 
the speed of the information flow, i.e., the speed of the averaging across the network, like with standard 
consensus and gossip algorithms, e.g., [11]. (The smaller r is, the faster the averaging is.) The next 
Theorem shows that distributed detection is asymptotically optimal if the network information flow is 
fast enough, i.e., if r is small enough. The Theorem also finds a lower bound on the rate of decay of 
the error probability, even when the sufficient condition for asymptotic optimality does not hold. Recall 
also Ctot in eqn. (16). 
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Theorem 6 Let Assumption 4 hold and consider the distributed detection algorithm defined by eqns. (18) 
and (19). Then, the following holds for the exponential decay rate of the error probability at each sensor: 



liminf -T;logi^ dis (£0 > { 



where 



Ctot if |logr| > |iV 2 (l + (l- ^)K) \SP\ 

~ {2W a Lp 2 + Tt m{ L~p) otherwise, 

{ 1 K , i v^^T^) 

K+l ' 



A K+l + 4' 

if lll^H < | logr| < |iV 2 (l + (1 - j^K))\\S% 

i J K 2 1 MMri _ Ik 
4 V ii s " , ii 4 ' 

if llogrl < illSWII . 



(50) 



Here K = (8m) / 1| S 11 \\ . Moreover, each sensor i is asymptotically optimal, and lim^oo — | log P[ dis (k) 
C to t, Vi, provided that: 



logr|>-iV 2 (l + (l--)^)||^ 



(51) 



x 10"' 




0.4 1r 0.6 



0.8 



Fig. 2. Lower bound on the exponential decay rate of the error probability (given by eqn. (52)) for the (generic) model in 
Section VI. The network has N = 20 sensors, and Ctot = 0.0047. The optimal rate (equal to Ctot) is achieved for p > 0.29. 

We proceed by proving Theorem 6. We first set up the proof and state some auxiliary Lemmas. We first 
consider the probability of false alarm, aj,dis(&) = (%i{k) > 0), but the same conclusions will hold 
for Pi,dis{k) = Pi (xi(k) < 0) also (See the Proof of Theorem 5.) We examine the family of Chernoff 
bounds on the probability of false alarm, parametrized by \i > 0, given by: 



kfMXi(k) 



--■C{kn), 



(52) 
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where Ej [a] := E[a\Hi], I = 0,1. We then examine the conditions under which the best Chernoff bound 
falls below the negative of the Chernoff information, in the limit as k — > oo. More precisely, we examine 
under which conditions the following inequality holds: 

limsup - logC(fc/i) < -C t ot- (53) 

In subsequent analysis, we will use the following Lemmas, Lemma 7 and Lemma 8; proof of Lemma 7 
is trivial, while proof of Lemma 8 is in the Appendix. 

Lemma 7 Let V be a N x N stochastic matrix and consider the matrix V = V — J. Then, for all 
i = 1, N, the following inequalities hold: 

N ~ N-\ 

Ei^i < 2 ^r <2 (54) 
i=i 

N ~ /VI 

El^l 2 < ^V<1- (55) 
i=i 

Lemma 8 Let Assumption 4 hold. Then, the following inequality holds: 

When proving Theorem 6, we will partition the time interval from the first to (k — l)-th time step 
in windows of width B, for some integer B > 1. That is, we consider the subsets of consecutive time 
steps {k - B, k - B + 1, k - 1}, {k - 2B, k - B - 1}, {k - (J - l)B, k - JB - 1}, 
{1, 2, k — JB — 1}, where J is the integer part of (Note that the total number of these subsets 
is J + 1; each of these subsets contains B time steps, except {1, 2, k — JB — 1} that in general has 
the number of time steps less or equal to B.) We then define the events Aj, j = 1, ...,J+ 1, as follows: 

B)\\<e} (57) 
>e}n {||$(fe,fe-2B)|| < e} 

>e}n{\\$(k,k-(j + l)B)\\ <e} 

(J - 1)B)|| > e} n fc - XB)|| < e} 

J£)||>e}. 



.4 



•J+l 



A/ 



|$(lfe,fc 
\&(k,k 

\${k,k 

\$(k,k 
\&(k,k 
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It is easy to see that the events Aj, j = 1, J + 1, constitute a finite partition of the probability space 
(i.e., each Ai and Aj, i / j, are disjoint, and the union of Aj's, j = 1, .., J" + 1, is 0.) 
By Lemma 8, the probability of the event Aj, j = 1, + 1, is bounded from above as follows: 

/V 4 

P (AO < ~^ rU ' 1)B - (58) 

We will next explain the idea behind using the partition A,-, j = 1, J + 1. First, we state Lemma 
9 that helps to understand this idea. 

Lemma 9 Consider the random matrices <&{k,j) given by eqn. (23), and fix two time steps, k and j, 
k > j. Suppose that the event Aj occurred. Then, for any s < k — jB, and for i = 1,...,N: 

N 

J2\Mk,s)]u\ < NVNe (59) 
l=i 

N 

J2\M k > s M 2 < Ne2 - (60) 
i=i 

Proof: See the Appendix. ■ 
Consider now the partition Aj, j = 1, J + 1. Given that Aj occurred, the matrices s), for 
s < k — jB, are "e-close" to J (in view of Lemma 9.) For s > k — jB, $(&, s) is not "e-close" to J, and 
thus we pass to the "worst case" as given by Lemma 7. (We may think of this as setting <£(£;, s) ps I.) 
Hence, given that ^ occurred, x(k), in a crude approximation, behaves as: 

<k)~\YlMi) + \ E 

/=1 l=k-jB+l 

Then, by conditioning x(/c) on j = 1, ...,J+ 1, and by using the total probability (as with Theorem 
5,) we express P[ dis (k) in terms of the j = 1, J + 1 "modes of decay" that correspond to Aj, 
j = 1, J + 1. Finally, we find the slowest among the J + 1 modes, as with Theorem 5. The difference 
compared to Theorem 5, however, is that here we work with the Chernoff bounds on a^dis^) and 
Pf<Xi s {k), rather than directly with ai^ s (k) and Pf A i s {k). After this qualitative description and after 
explaining the main idea behind the calculations, we proceed with the exact calculations, some of which 
are moved to the Appendix. 

Proof of Theorem 6: Consider the Chernoff bound on di^ s {k) given by eqn. (52), and denote by 
mo := maxj=i ) ... ) jv We first express the Chernoff upper bound on limsup^^ | logPj e dis (A;) in 

terms of the "modes of decay," similarly as in the Proof of Theorem 5. Namely, it can be shown (details 
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are in the Appendix) that the following inequality holds: 

limsupilogc^dis^) = limsup- log P? dis (£:) (61) 

< limsup- log C(/c/x)£(fc/z), V/i > 0, Ve € (0, 1), VP = 1,2,... 



where 



C(fc M ) = exp((/c-l)(^V + lm(° ) / u)) (62) 

J _ 
tf(]fe M ) = J^exp (|2/i 2 -^|m |2(j + l)fi + (fc - (j + l)B)JVVWe}) (63) 

exp II^H {(j + 1)5 + (k - (j + 1)5)^}) (- 



' TV 4 r jS 



Notice that the summands in eqn. (63) are the "modes of decay" that correspond to the events Aj+\, 
j = 1, ...,J + 1. Next, after bounding from above by "the slowest mode," we get: 

8{kfi) < (J + l) max exp (\2fi 2 - n\ m \2(j + l)B + (k - (j + l)B)Ny/Ne\) (64) 
j=o,...,j VI J / 

exp^H^II {(j + l)B + (k-(j + l)B)Ne 2 f) {^^j ■ (65) 

Introduce the variable 6 = 9(j) = 3 -^. We now replace the maximum over the discrete set j = 0, J, 
in eqn. (64) by the supremum over j £ [0, J], i.e., 6 G [0, 1]; we get: 

8{kn) < (J + l) sup exp (\2fi 2 -fi\m \2k9(l- N^/Ne) + B(2 - N VN e) + kN^/N e})(66) 
9e[o,i] v L )J 

i 2 „_„ r , ^ „ 9s , 91 \ /N A r> By 



exp ^ {fc0(l - iVe 2 ) + 5(1 - iVe 2 ) + ke 2 }^j 



e 2 



Taking the limsup as k — > oo, and then letting e — > 0, we obtain the following inequality: 

limsup - log C(fc/i) < sup (f>(9; fi), > (67) 

fe^oo 0€[O,1] 

^M) = ^a 2 Ll x 2 + ^mf ^ + 2\2^-^\m9 + ^\\S^\\9 + 9\ogr. (68) 

Eqn. (67) gives a family of upper bounds, indexed by \i > 0, on the quantity lim sup^^ \ log ai,dis(fc); 
we seek the most aggressive bound, i.e., we take the infimum over fi > 0. 

We first discuss the values of the parameter r, for which there exists a range fi G [\,~p\ (where ~p is 
given by eqn. (50),) where the supremum in eqn. (67) is attained at 6 = 0. It can be shown that this 
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range is nonempty if and only if: 

|logr| > 1 \\S r >\\. (69) 
8 

In view of eqns. (67), (68), and (69), we have the following inequality: 

limsup - logC'(fyi) < rmn { tt^^ 2 + ii 1 . (70) 

The global minimum (on n e M) of the function /i >->■ ^W^li 2 + j^m^ jx is attained at /x* = N/2; and it 
can be shown that it equals the negative of the Chernoff information — C to t = — \{ m i — mo) T <S l-1 (mi — 
mo). Thus, a sufficient condition for limsup^^ | log cti^^k) < limsup^^ \ logC'(kfi) < — C to t is 
that li* = y e - By straightforward algebra, it can be shown that the latter condition translates into 
the following condition: 

|logr| > l -N 2 (l + {l-^)K^ \\S^\\. (71) 

The analysis of the upper bound on aj dis(^) remains true for the upper bound on ^^{k); hence, 
we conclude that, under condition (71), we have: limsup^^ \ log f3i^\ s {k) < —Ctot- Thus, under the 
condition (71), the following holds: 

limsup ^ log Pf Ais { k) = max (limsup ^ log a i)dis ( k), limsup ^ log f3 itdis ( kU < -C to t- (72) 

On the other hand, by the Chernoff Lemma (Lemma 3,) we also know that 

Uminfilog^ dis (fe) > -C tot . (73) 

By eqns. (72) and (73), we conclude that lim^oo I log Pf dis (k) = — C to t> for the values of r that satisfy 
the condition (71). Hence, condition (71) is a sufficient condition for asymptotic optimality. 
Consider now the values of r such that: 

< | logr| < ^N 2 (l + (1 - ^)k) (74) 

In this range, \ < Jl < fi* = N/2, and the minimum of the function ll >->■ i^pul/j. 2 + j^m^ ^ on 
G [|, A*] is attained at ll = Ji. Hence, for the values of r in the range (74), we have the following 
bound: 

limsup i log Pi )dis (k) = max (limsup * log a i;dis ( A;), limsup * log ft,dis(») (75) 

1 2-2 1 (0)_ 

" 2~N~ 2 ° Ll1 + N m L fl - (76) 
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Finally, consider the values of r such that the condition in (69) does not hold, i.e., §||<S ,?? || > | logr|. 
We omit further details, but it can be shown that, in this case, the supremum in eqn. (67) is attained at 
6 = only for the values of fi in a subset of [0, \\; and it can be shown that limsup^^ | log P i e dig (/c) 
can be bounded as given by eqn. (50). ■ 

B. Necessary condition for asymptotic optimality 

We proceed with necessary conditions for asymptotic optimality, for the case when the sensors' 
observations are spatially uncorrelated, i.e., the noise covariance S = Diag [af, u^). Denote by 
Ei(k) the event that, at time k, sensor i is connected to at least one of the remaining sensors in the 
network; that is, 

Ei(k) := { max W^k) > ol . (77) 

[j=l,...,N, j^i J 

Further, denote by Pi(k) = Pi = F(Ei(k)) We have the following result. 

Theorem 10 (Necessary condition for asymptotic optimality) Consider the distributed detection algorithm 
in eqns. (18) and (19) with spatially uncorrelated sensors' observations, and let Assumption 4 hold. Then, 
a necessary condition for the asymptotic optimality of distributed detection at sensor i is: 

|log(l-Pi)| >Ctot-Ci. (78) 

Proof: Consider ai^is(k), but remark that the same conclusions will hold for /3j i dis(^), as o^disW = 
A,dis(^)- (See the proof of Theorem 5.) Further, using inequalities (7), we have: 

Oi,dis(fc) = F o (xi(k)>0) (79) 

> p (zi(fc) > o| n s fe =1 Ei(sf) p (nju^t*)) (80) 



Thus, we have that 



liminf I loga ljdis (fc) > - (Q + | log(l - P)\) . (82) 



On the other hand, we know that 



liminf- log f£ dis ( A;) > liminf - log a ijdis ( k) > -C tot . (83) 



The claim of Theorem 10 now follows from eqns. (82) and (83). 
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VII. Simulations 

In this section, we corroborate by simulation examples our analytical findings on the asymptotic 
behavior of distributed detection over random networks. Namely, we demonstrate the "phase change" 
behavior of distributed detection with respect to the speed of network information flow, as predicted 
by Theorem 6. Also, we demonstrate that a sensor with poor connectedness to the rest of the network 
cannot be an optimal detector, as predicted by Theorem 10; moreover, its performance approaches the 
performance of an isolated sensor, i.e., a sensor that works as an individual detector, as connectedness 
becomes worse and worse. 

Simulation setup. We consider a supergraph G with N = 40 nodes and M = 247 edges. Nodes are 
uniformly distributed on a unit square and nodes within distance less than a radius r are connected by 
an edge. As averaging weights, we use the standard time- varying Metropolis weights Wy(fc), defined 
for {i,j} G E,i / j, by Wij(k) = 1/(1 + max(dj(fc), dj(k))), if the link {i,j} is online at time k, 
and otherwise. The quantity di(k) represents the number of neighbors (i.e., the degree) of node i at 
time k. Also, Wu(k) = 1 - £jen 4 (k) Wij(fc), for all i, and Wij(k) = 0, for i / j, {i,j} £ E. The 
link failures are spatially and temporally independent. Each link {i,j} € E has the same probability of 
formation, i.e., the probability of being online at a time, q^ = q. This network and weight model satisfy 
Assumption 4. 

We assume equal prior probabilities, P(Hq) = F(H{) = 0.5. We set the N x 1 signal vector under 
H\ (respectively, Ho) to be mi = 1 (respectively, mo = 0.) We generate randomly the covariance 
matrix S, as follows. We generate: a N x N matrix Ms, with the entries drawn independently from 
U[0, l]-the uniform distribution on [0, 1]; we set Rs = M^Mj; we decompose R$ via the eigenvalue 
decomposition: Rs = QsA-sQ~s'i we generate a N x 1 vector us with the entries drawn independently 
from J7[0, 1]; finally, we set S = as QsDi&g(us)Q~s, where as > is a parameter. For the optimal 
centralized detector, we evaluate -P c 6 cn (/c) by formula (6). For the distributed detector, we evaluate Pf dis {k) 
by Monte Carlo simulations with 20,000 sample paths (20,000 for each hypothesis Hi, I = 0, 1) of the 
running consensus algorithm. 

Exponential rate of decay of the error probability vs. the speed of information flow. First, we 
examine the asymptotic behavior of distributed detection when the speed of network information flow 
varies, i.e., when r varies. (We recall that r := A 2 (E [W(k) 2 ]).) To this end, we fix the supergraph, 
and then we vary the formation probability of links q from to 0.75. Figure 3 (bottom left) plots the 
estimated exponential rate of decay, averaged across sensors, versus q. Figure 3 (bottom right) plots 
the same estimated exponential rate of decay versus 1 — r. We can see that there is a "phase change" 
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behavior, as predicted by Theorem 6. For q greater than 0.1, i.e., for 1 — r > 0.25, the rate of decay of 
error probability is approximately the same as for the optimal centralized detector C to t-the simulation 
estimate of C to t is 0.0106. 3 For q < 0.1, i.e., for 1 — r < 0.25, detection performance becomes worse 
and worse as q decreases. Figure 3 (top) plots the estimated error probability, averaged across sensors, 
for different values of q. We can see that the curves are "stretched" for small values of q; after q exceeds 
a threshold (on the order of 0.1,) the curves cluster, and they have approximately the same slope (the 
error probability has approximately the same decay rate,) equal to the optimal slope. 
Study of a sensor with poor connectivity to the rest of the network. Next, we demonstrate that a sensor 
with poor connectivity to the rest of the network cannot be an asymptotically optimal detector, as predicted 
by Theorem 10; its performance approaches the performance of an individual detector-sensor, when 
its connectivity becomes worse and worse. For i-th individual detector-sensor (no cooperation between 
sensors,) it is easy to show that the Bayes probability of error, P? no coopcr .(fc) equals: Pf no coopcr (k) = 
Q (V% ™';~ ;;;;;; ) , where m;, no cooper. = \ [ ^, and <J? nocooper = It is easy to s how that the 

Chernoff information (equal to Hindoo ^ log Pf no cooper {k)) for sensor i, in the absence of cooperation, 
is given by g 1 ^^- 

We now detail the simulation setup. We consider a supergraph with N = 35 nodes and M = 263 
edges. We initially generate the supergraph as a geometric disc graph, but then we isolate sensor 35 from 
the rest of the network, by keeping it connected only to sensor 3. We then vary the formation probability 
of the link {3,35}, (73,35, from 0.05 to 0.5 (see Figure 4.) All other links in the supergraph have the 
formation probability of 0.8. Figure 4 plots the error probability for: 1) the optimal centralized detection; 
2) distributed detection at each sensor, with cooperation (running consensus;) and 3) detection at each 
sensor, without cooperation (sensors do not communicate.) Figure 4 shows that, when 93 35 = 0.05, 
sensor 35 behaves almost as bad as the individual sensors, that do not communicate (cooperate) with 
each other. As q increases, the performance of sensor 35 gradually improves. 

VIII. Conclusion 

We studied distributed detection over random networks when at each time step k each sensor: 1) 
averages its decision variable with the neighbors decision variables; and 2) accounts on-the-fly for its new 
observation. We analyzed how asymptotic detection performance, i.e., the exponential decay rate of the 
error probability, depends on the random network connectivity, i.e., on the speed of information flow across 

3 In this numerical example, the theoretical value of C to t is 0.009. The estimated value shows an error because the decay of 
the error probability, for the centralized detection, and for distributed detection with a large q, tends to slow down slightly when 
k is very large; this effect is not completely captured by simulation with k < 700. 
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Fig. 3. Monte Carlo estimate of the performance of distributed detection for different values of the link formation probability q. 
Top: Error probability averaged across N sensors. Each line is labeled with the value of q; performance of centralized detection 
is plotted in gray. Bottom left (respectively, right): Estimated exponential rate of decay of the error probability vs. q (respectively, 
1 — r). Recall that r := A 2 (E [W^fc) 2 ]). 



network. We showed that distributed detection exhibits a "phase change." Namely, distributed detection 
is asymptotically optimal, if the network speed of information flow is above a Chernoff information 
dependent threshold. When below the threshold, we find a lower bound on the achievable performance (the 
exponential rate of decay of error probability,) as a function of the network connectivity and the Chernoff 
information. Simulation examples demonstrate our theoretical findings on the asymptotic performance of 
distributed detection. 
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Fig. 4. Error probability averaged across sensors for the optimal centralized detection, distributed detection at each sensor 
(with cooperation), and detection at each sensor, without cooperation. The formation probability 93,35 of the link {3, 35} varies 
between 0.05 and 0.5: g 3 ,3 5 =0.05 (top right); 0.2 (top left); 0.3 (bottom left); 0.5 (bottom right). 



Appendix 

A. Proof of inequalities (43) 

Recall x(l\ k) in eqn. (38). Using eqn. (39), and noting that (1 — p)^ 1 > p(l — p) , we bound 
&i,dis(k) from below as follows: 

a lA]s (k) > max Q( X (l; k)) p(l - p)*"'" 1 (84) 
{=0,...,re— 1 

1 ^\ e -\x(iMY p{l _ p) k-i-i 



> max ,„ ,™ e ~ iXW ) P0--P) (85) 
l=o,...,k-l l + x{hk) 2 

> _L max e-i^ fc ) 2 p(l-p) fc - z - 1 (86) 
~ y/2n 1 + 2kNCi l=o,...,k-l r\ ?i \ > 

= , max pe +( ,( <=> (87) 

2tt 1 + 2kNd i=o,..„k-l 

max pe +l ' * (88) 



2tt 1 + 2/cTVCi j=o,...,fc-i 

,L max pe -**C**>. (89) 

/27r 1 + 2/ciVCi j=o,...,k-i F 
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Inequality in eqn. (85) is due to (7), equality in (88) is by letting j = k — I — 1, and equality in (89) is 
by definition of cf)(j; k) in eqn. (41). Taking lim inf k^oo \ logo^dis^)* an ^ using (33), (84)-(89) yields: 

liminf - logPf dis (A;) > liminf - logaj dis(^) > — limsup 1 min (f)(j;k)\ . (90) 

fc^oo k ' fc^oo k ' fe— s>oo [ j=0,...,k— 1 J 

Next, using eqn. (39), we bound Po {xi(k) > 0) from above as follows: 

cWfc) < k max Q( X (l; k)) (1 - pf' 1 ' 1 (91) 

1=0,. ...k— 1 



< -)= max -T^rve-^^n-p)^- 1 
V^7r i=o fc-i x(l;k) 



(92) 

< k^=^L= max e-lxCWri-p)*-'- 1 (93) 
^2Qfc l=o fc-i 1 F; 



r 1 1 , -(fe-i-l)|log(l- P )| 

fc . . max e +( )( t>> (94) 

V2vf V2C ; fc ;=o,...,fc-i 



1 1 fcCtot i -j|log(l-p)| 

k^-^= max e ^-^i m (95) 

^2^ V2Cifc i=o,...,fc-i 

k-^— L= max e- k< M*). (96) 
V2vr V2Cifc j=o,...,fc-i 



Inequality in (92) is due (7), and equality in (96) is by definition of 4>(j; k) in eqn. (41). We complete 
the proof by noting that equality in (32) and inequalities (91)-(95) imply that: 

lim sup — log PfdiJk) = lim sup — log a.{ dis(^) < — nm m f I mm 4>{j\ r • (97) 

k^oo k ' fe^oo fc fc^oo [jg{ ,...,fc-l} J 



B. Proof of Lemma 8 

Consider the random vector z^(k,j) = $(fc, j)e,, which is equal to the z-th column of the matrix 
&(k,j). First, by the Markov inequality, we have: 



P (\\z^(k,j)\\ > e) < j) T z«(£;, j) 



(98) 



Now, we can bound E [z^\k, j) T z^(k, j)] as follows: 



E 



zW(k,j) T z®(k,j) 



= E 

= E 

< ||E 

= rE 



E 



e[$(k - l,j) T W(k - l)W(k - l)$(k - l,j) ei \W(j), ...,W(k - 2) 



e- <fr(k - E W(fc-l)W(fc-l) $(ife-l,j)e i |W(j),-,W , (fe-2) 



W(fc- 1) 



E 



eJ<5>(k-l,j) T <S>(k-l,j) ei 



>)(fe-i,j)V < )(fc-i,^ 



(99) 
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Now, bounding successively E [z^\k — s, j) T z^\k — s, j)] for s = l,...,k — j, as in eqn. (99), we 
obtain: 



E 

which yields (by eqn. (98):) 



z®(k,j) r z®(k,j) 



< r k ~ j , (100) 



F^ l \k,j)\\>e) <^r k ^,z = l,...,N. 
Now, observe that ||^W(fc, j)||i < N\\z^(k,j)\\, and hence, 

¥(\\z^(k,j)\\ 1 >e)<F{\\z^(k,j)\\>^). 
Further, in view of the matrix norm inequality \\B\\ < y/~N\\B\\i, we have: 
p(||$(fc,j)|| >e) < P (W,j)lli > =p(. maxjz^ikj)^ 

= p (u^ii^Mlli > < f> > ^) < 



(101) 



e 

> 



C. Proof of Lemma 9 

First, note that, if Aj occurred, we have that: 

\mk,s)\\<e, Vs<k-jB, 

since \Mk,s)\\ = \\[3>(k,k-jB)$(k-jB-l,s)\\ < \\&(k,k — jB)\\\\®(k — jB — l,s)\\, and ||[$(fc,fc- 
jB)\\ < e and ||$(fe - jB - 1, s)|| < 1. Inequality (59) holds true, because: 

AT 

V|[$(My < N max J [*(fc,s)]«|> 
^ M=i,...,JV 

and maxj ; =li jv|[$(fc, < a/TV ||$(fc, s)|| < \/JVe. Inequality (60) holds true, because, for s < 
k - jB, 

N 

|[*(*. s)]u\ 2 < ||*(*. s)f F < N\\*(k, s)f = Ne 2 . 

l=i 

D. Proof of eqn. (61) 

Consider the Chernoff bound on a^^k), given by: 



C {kfj) := Eo [exp (kfi Xi(k))] = Eo exp 

where A = jj, e,, A € 



(102) 
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Further, we have: 

C(kfi) 



En 



En 



fc-i 



exp [ X T Y J Hk,j)vU)+ A T r?(fc) 
3=1 



(103) 



fe-i 

exp | \ T ^2<S>(k,j)r](j) 

3=1 



E n 



exp ( X T r](k) 



where the last equality holds because rj(k) is independent from rj(j) and W(j), j = 1, k — 1. We will 
be interested in computing limsup^^ ^A^(A;/x), for all with this respect, remark that 



lim — log Eo 

k— >oo k 



exp ( X T r](k) 



= 0, 



for all A 6 E , because ry(A;) is a Gaussian random variable and hence it has finite log-moment generating 
function at any point A. 

Thus, we have that rimsupj^^-, \ logC (k/j.) = limsup^^ \ logC'(fc^), where 

(fe-i 
\ T J2<S>(k,j)r)(j) 

We thus proceed with the computation of C'(kfi). Conditioned on VF(1), W(2), W(k — 1), the random 
variables \ T &(k, j)r)(j), j = 1, k — 1, are independent; moreover, they are Gaussian random variables, 
as linear transformation of the Gaussian variables r](j). Recall that and S v denote the mean and 
the covariance of 7](k) under hypothesis Hq. After conditioning on W(l), W(2), W(k — 1), using the 
independence of and r](s) s / j, and using the expression for the moment generating function of 
r](j), we obtain successively: 



C'(m) 



E 



E 
E 



En 



En 



fe-i 



exp A T ^$(/c,i)r/(j) 



|W(l),...,W(fc-l) 



o [nJ- 1 1 exp(A T $(A;,j)r ? 0))] |W(1), .., W(fc - ] 
n^E exp (A T $(fc,i)my))exp Qa t <&(£:, j) T 5"$(fe, j)A 
E[nyexp(A T + m(°)) 

exp Q (j + Hk,j)) T S r > (j + $(fe, j)))]- 
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Denote further: 

5(kn) := E[exp ( A T ) exp ( ^A T ( j) r Sr>*(k,j) I A ) (104) 



fc-i 



fc-i 



fc-l 



exp |^ A T JS^ <!>(£:, j) A j exp ^A T ^ $(fc,j) T ^JA ) ], 
C(fcAi) := exp ^(jfe - 1) (V Jmf - ^A T JA^j ^ , 



(105) 



where dependence on H$ is dropped in the definition of 8(kfi). Then, it is easy to see that C'(kfj,) = 
C(kn)S(kfi). 

Recall the expressions for v, and a\ in eqns. (10), (12). After straightforward algebra, it can be 
shown that C(k/j,) equals the expression in eqn. (62). Remak further that 5(k/j,) equals: 



S(kn) = E 



fc-i 



exp (^-^eJYsHkJ)™? exp ^-ej £ *{k,j)Sr>Q{k, j) T e 



j'=i 



Recall that mo := maxj = i v .. ; ^ |[m^]j|. Now, 5{kji) can be bounded from above as follows: 



fc-i 



(106) 



(5(ife/i) < E 



< E 



fc-l TV 



exp ( l^-^lmo^^ l[*(M]«l ] exp ( ^||Sl£>(M]i 



fc-i 



fc AT 



exp [ |/x- M 2 |m ^^ l[*(M]«l ] exp [ ^ \\sr>\\ £ | [*(*, j)] u 



j'=i 
fc 



(107) 



j'=i 



Using Lemmas 9, 8 and 7, using the total probability with the partition Aj, j = 1, J + 1, we get 
the inequality (61). 
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